This invention relates generally to data input to devices and, more particularly, to finger-free operation of electronic devices.
Advances in electronics technology are leading to ever-smaller device designs with an ever-increasing variety of applications. Many xe2x80x9ccomputingxe2x80x9d devices, including handheld computers, Personal Digital Assistants (xe2x80x9cPDAxe2x80x9d), pagers and cellular phones, are now small enough to be easily carried or even wom.
Highly miniaturized computing devices, such as handheld computers and PDAs, have significant operational limitations as a consequence of their small size. In particular, the computer-human interface is highly constrained: the small size greatly restricts possible methods for data input. Traditional desktop computer-human interfaces, those that rely on keyboards and pointing devices, translate poorly to very small devices. Typically, PDAs lack a keyboard. Some handheld computer designs attempt to mimic desktop computer designs, however, only a miniature keyboard can be included.
In response to this problem, these computing devices generally provide for data input through a stylus and/or a limited number of buttons. Other miniature portable devices, such as pagers and cellular phones, typically rely on buttons for command input.
Because of the ease and familiarity of the graphical user interface (xe2x80x9cGUIxe2x80x9d) typically employed in the display of data in modern computers, for some applications it is desirable to maintain some features of a GUI in a miniature, computing device. The typical GUI relies on the computer user""s input of a few command types, submitted via a pointing device and button clicks, to generate a response in the GUI. For example, pointing at a scroll bar and clicking can cause scrolling through a document or list; items in the list or in a graphical display can be selected by again pointing and clicking; and an item, such as an Internet address or a program, can be activated by pointing and double-clicking. As noted, however, command submission to the GUI in a miniature device cannot be accomplished with a xe2x80x9cmousexe2x80x9d point and click approach.
Newer approaches to the computer-human command submission interface have the goal of improving the interface for computers of all sizes. One approach is to employ tilt and/or translation sensors to achieve an alternate means of data input. In xe2x80x9cInertial Proprioceptive Devices: Self-Motion-Sensing Toys and Toolsxe2x80x9d by C. Verplaetse, IBM Systems Journal, Vol. 35, pg. 639 (1996), the author describes devices that are able to sense and react to their own position and orientation through use of embedded inertial motion sensors. The author further describes commercially available accelerometers and gyroscopes for use as inertial motion sensors in portable computing devices.
An implementation of the position and orientation sensing approach to data input is described in xe2x80x9cDesign of Spatially Aware Graspable Displaysxe2x80x9d, by David Small and Hiroshi Ishii, CHI 97, pg. 367. The authors describe a large handheld or table positioned device that can be used to read a xe2x80x9cnewspaperxe2x80x9d or other written document. Tilting of the device, in an intuitive manner, provides input commands for scrolling up or down, and left or right through the document: the graphical display responds to the angular orientation of the device. Translating the device closer to or further from the body of the user controls zooming of the document, in effect, scrolling in a third dimension. A button provides a xe2x80x9cclutchxe2x80x9d to lock and unlock the response of the display to user commands.
A much smaller prototype handheld device that also makes use of tilt for data input is described in xe2x80x9cTilting Operations for Small Screen Interfaces (Tech Note)xe2x80x9d by Jun Rekimoto, UIST ""96, pg. 167 (1996). This device employs a combination of: 1) tilting for input of scrolling commands to move through text or an image; and 2) button pushing for input of activating commands for navigation through the display of text or to view a portion of an image in greater detail.
The above described prototype data input methods still have the problem of requiring use of fingers for some types of command input, typically via buttons or a stylus. What is desired is a method for completely finger-free data input.
A method is provided for controlling a handheld computing device through the use of gesture commands. A gesture command corresponds to a pattern of movement of the handheld device that is measured as a function of time F(t), where ts is a start time and tf is a finish time such that the interval of time is tf minus ts. The gesture command further corresponds to angular orientations of the handheld device.
The method comprises the steps of: measuring movement of the device over an interval of time with a motion sensor mounted to the device, the measuring producing a signal; analyzing the signal measured over the interval of time to determine a gesture command that corresponds to the movement of the device; and processing the gesture command to control the handheld computing device. To enable use of different gesture commands, the method includes a comparison of the signal to a catalog of gesture commands to determine a particular gesture command.
A great range of gesture commands is possible given the use of different axes and angular directions of rotation for a variety of different patterns of movement. Use of gesture commands can be combined with use of position commands to further extend the range of commands available for control of a computing device. In this manner, a computing device can be controlled without the use of buttons for command input.